Combining metrics for software evolution assessment by means of Data Envelopment Analysis

Research and practice in software engineering have led to an extensive set of metrics for the evaluation of almost every aspect of software development. One of the major challenges for any quality model is the combination of metrics, which are complementary to each other. In this paper, we propose the use of Data Envelopment Analysis (DEA), a non-parametric technique employed in economics, as a means of providing a unified view of selected design metrics. The proposed application of DEA aims at assessing the overall trend of quality during the evolution of software systems, by considering releases of a project as units to be ranked. An important benefit derived from the use of DEA is the ability to “normalize” the evaluation over the size characteristics of the examined systems, which is vital when comparing projects of different scale. Results are presented for successive versions of two open-source, one industrial and one research project, whereas validation, whereas validation is performed by comparing the findings with the results obtained by Analytic Hierarchy Process, which is an acknowledged multi-criteria decision analysis approach. According to the results, DEA enables the perception of global trends in qualitative characteristics, which would be otherwise difficult to recognize and interpret. Copyright © 2012 John Wiley & Sons, Ltd.